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Abstract. We consider the model checking problem for probabilistic pushdown automata 
(pPDA) and properties expressible in various probabilistic logics. We start with properties 
that can be formulated as instances of a generalized random walk problem. We prove that 
both qualitative and quantitative model checking for this class of properties and pPDA 
is decidable. Then we show that model checking for the qualitative fragment of the logic 
PCTL and pPDA is also decidable. Moreover, we develop an error-tolerant model checking 
algorithm for PCTL and the subclass of stateless pPDA. Finally, we consider the class of 
a;-regular properties and show that both qualitative and quantitative model checking for 
pPDA is decidable. 



1. Introduction 

Probabilistic systems can be used for modeling systems that exhibit uncertainty, such 
as communication protocols over unreliable channels, randomized distributed systems, or 
fault-tolerant systems. Finite-state models of such systems often use variants of probabilistic 
automata whose underlying semantics is defined in terms of homogeneous Markov chains, 
which are also called "fully probabilistic transition systems" in this context. For fully 
probabilistic finite-state systems, algorithms for various (probabilistic) temporal logics like 
LTL, PCTL, PCTL*, probabilistic ^-calculus, etc., have been presented in |LS82[ IHS84I 
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EEEB EyHH IHJ941 lASR+951 ICYOKl lHK97l ICSSnsj. As for infinite-state systems, most 
works so far considered probabilistic lossy channel systems |IN97j which model asynchronous 
communication through unreliable channels |BE99l lABIJOSi IAR03I IBS03j . A notable recent 
result is the decidability of quantitative model checking of liveness properties specified by 
Biichi- automata for probabilistic lossy channel systems |Rab03j . In fact, this algorithm is 
error tolerant in the sense that the quantitative model checking is solved only up to an 
arbitrarily small (but non-zero) given error. 

In this paper we consider probabilistic pushdown automata (pPDA), which are a natural 
model for probabilistic sequential programs with possibly recursive procedure calls. There is 
a large number of results about model checking of non-probabilistic PDA or similar models 
(see for instance |AEY011 155971 lEHRSOOl IWalOlj ^. but the probabilistic extension has so 
far not been considered. As a related work we can mention |M098j . where it is shown 
that a restricted subclass of pPDA (where essentially all probabilities for outgoing arcs are 
either 1 or 1/2) generates a richer class of languages than non-deterministic PDA. Another 
work |AMP99j shows the equivalence of pPDA and probabilistic context-free grammars. 
There are also recent results of |BKSn5| IEYn5| lEYj which are directly related to the results 
presented in this paper. A detailed discussion is postponed to Section El 

Here we consider model checking problems for pPDA and its natural subclass of stateless 
pPDA denoted pBPA^ and various probabilistic logics. We start with a class of properties 




DDZ DZ Z IZ IIZ 

Figure 1: Bernoulli random walk as a pBPA 

that can be specified as a generalized random walk problem. To get a better intuition 
about this class of problems, realize that some random walks can easily be specified by 
pBPA systems. For example, consider a pBPA with just three stack symbols Z, I, D and 
transitions Z IZ, Z DZ, I —> II, I ^—^ e, D ^—^ DD, and D e, where x £ [0, 1] 
and e denotes the empty string. A transition X w means that if the current top stack 
symbol is X,then it can be replaced by w with probability x. The transition graph of this 
pBPA with Z as initial stack content (see Fig. ^ is the well-known Bernoulli walk. A typical 
question examined in the theory of random walks is "Do we eventually revisit a given state 
(with probability one)?", or more generally "What is the probability of reaching a given 
state from another given state?" For example, it is a standard result that the state Z of 
Fig. His revisited with probability 1 iff x = 1/2. This simple example indicates that answers 
to qualitative questions about pPDA (i.e., whether something holds with probability 1 or 
0) depend on the exact probabilities of individual transitions. This is different from finite- 
state systems where qualitative properties depend only on the topology of a given finite-state 
Markov chain |II,T94j . 

The generalized random walk problem is formulated as follows: Let Ci and C2 be subsets 
of the set of states of a given Markov chain, and let s be a state of Ci. What is the probability 
that a run initiated in s hits a state of C2 via a path leading only through the states of Ci? 



This is a standard notation adopted in concurrency theory. The subclass of stateless PDA corresponds 
to a natural subclass of ACP known as Basic Process Algebra IBW90I . 
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Let us denote this probability by V{s,CiU €2)- The problem of computing V{s,CiU C2) 
has previously been considered (and solved) for finite-state systems, where this probability 
can be computed precisely |HJ941 ir)Y95j . In Sectional we propose a solution for pPDA 
applicable to those sets Ci,C2 which are regular, i.e., recognizable by finite-state automata 
(realize that pPDA configurations can be written as words of the form pa, where p is 
a control state and a a sequence of stack symbols). More precisely, we show that the 
problem whether V{s,CiU C2) ~ g, where ~ S {<,<,>,>,=} and g G [0,1], is decidable. 
Interestingly, this is achieved without explicitly computing the probability V{s,CiU 02)- 
Moreover, for an arbitrary precision < A < 1 we can compute rational lower and upper 
approximations G [0, 1] such that < V{s,CiUC2) < V and - < A. 

In Section |1J we consider the model checking problem for pPDA and the logic PCTL. 
This is a more general problem than the one about random walks (the class of properties 
expressible in PCTL is strictly larger). In Section [4. 11 we give a model checking algorithm 
for the qualitative fragment of PCTL and pPDA processes. For general PCTL formulas and 
pBPA processes, an error tolerant model checking algorithm is developed in Section 14.21 
The question whether this result can be extended to pPDA is left open. 

Finally, in Section|Slwe prove that both qualitative and quantitative model checking for 
the class of cj-regular properties is decidable for pPDA. In |EKMn4] . it was shown that the 
qualitative and quantitative model-checking problem is decidable for pPDA and a subclass 
of (j-regular properties that are definable by deterministic Biichi automata. Later, it has 
been observed in |BKSn5j that the technique can easily be generalized to Muller automata, 
and thus the decidability result was extended to all w-regular properties (in |BKSn5j , some 
complexity results were also presented). The construction presented in this paper is a 
slightly generalized and polished version of the algorithms given in |EKMn4l IBKSn5j , which 
can now be seen as instances of a more abstract result. 

In Section El we conclude by remarks on open problems and recent related work of 
[HKSnRLlKYTIKllRYj . 

2. Preliminary Definitions 

Definition 2.1. A probabilistic transition system is a triple T = (S,^, Prob) where /S is a 
finite or countably infinite set of states, ^ C S x S is a transition relation, and Prob is a 
function which to each transition s ^ i of T assigns its probability Prob{s — > i) G (0, 1] so 
that for every s G S we have 

^^Probis^t) G {0,1} 

The sum above is iff s does not have any outgoing transitions. 

In the rest of this paper we also write s ^ t instead of Prob{s — > t) = x. A path in 
T is a finite or infinite sequence w = sq; si; ■ • • of states such that Si — > Sj+i for every i. 
We also use w^i) to denote the state Si of w (by writing w{i) = s we implicitly impose the 
condition that the length of w is at least i + A run is a maximal path, i.e., a path which 
cannot be prolonged. The sets of all finite paths, all runs, and all infinite runs of T are 
denoted FPath, Run, and IRun, respectively^. Similarly, the sets of all finite paths, runs, 
and infinite runs that start in a given s G S are denoted FPath{s), Run{s), and IRun{s), 
respectively. 



'In this paper, T is always clear from the context. 
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Each w G FPath determines a basic cylinder Run{w) which consists of aU runs that 
start with w. To every s G S we associate the probabihstic space {Run{s),J^,V) where 
T is the o"-field generated by all basic cylinders Run{w) such that w starts with s, and 
"P : ^ — > [0, 1] is the unique probability function such that V{Run{w)) = n™Q^Xj where 
w = sq]- ■ ■ ]Sm and Sj ^ Sj+i for every < i < m (if m = 0, we put V{Run{w)) = 1). 

2.1. The Logic PCTL. PCTL, the probabilistic extension of CTL, was defined in jH.IMj . 
Let Ap = {a, b,c, . . .} be a countably infinite set of atomic propositions. The syntax of 
PCTL^ is given by the following abstract syntax equation: 

::= tt I a I -193 \ (fi A (p2 \ X^^^p \ ifi U ^^^2 

Here a ranges over Ap, q G [0,1], and ~ G {<,<,>,>}• Let T = {S,^, Prob) be a 
probabilistic transition system. For all s £ S, all C,Ci,C2 Q S, and all A; G No, let 

• Run{s,XC) = {we Run{s) \ w{l) G C} 

• Run{s,CiU C2) = {tf G Run{s) | 3i > : w{i) G C2 and w(j) G Ci for all < j < i} 

• FPath^{s,CiUC2) = {so;---,Si G FPath{s)\0 < £ < /c,s^ G C2 and sj G Ci\C2 for all < 

• FPath{s, CiUC2) = (J^o ^^a^^'' («, Ci C2) 

The set Run{s,XC) is clearly "P-measurable, and the same holds for Run{s,CiU C2) because 
V{Run{s,CiUe2)) = V{Run{w)). 

weFPath{s,CiUC2) 

In the rest of this paper, we will usually write V{s,XC) and V{s,CiU C2) instead of 
V{Run{s, XC)) and V (Run {s, Ci U C2)), respectively. 

Let u : Ap ^ 2*^ be a valuation. The denotation of a PCTL formula ip over T w.r.t. v, 
denoted {(fY , is defined inductively as follows: 

[ttf = S 

laf = v{a) 

lA'-vr = {s(iS\v{s,xyf)^e] 
yiU-^M" = {s(^s\v{s,i^^ruy2r)-Q} 

As usual, we write s \='^ ip instead of s G \ipY . 

The qualitative fragment of PCTL is obtained by restricting the allowed operator/ 
number combinations to '< 0' and '> 1', which will be also written as '= 0' and '= 1', 
resp. (Observe that '< 1', '> 0' are definable from '< 0', '> 1', and negation; for example, 
aU<^b = ^{aU^^b).) 

2.2. Probabilistic PDA. 

Definition 2.2. A probabilistic pushdown automaton (pPDA J is a tuple A = (Q, F, 5, Prob) 
where Q is a finite set of control states, F is a finite stack alphabet, (JCQxFxQxF* 
is a finite transition relation (we write pX — > qa instead of {p,X,q,a) G 5), and Prob is a 



'For simplicity we omit the bounded 'until' operator of IHJ94| . 
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function which to each transition pX qa assigns its probabiUty Prob{pX — > qa) G (0, 1] 
and satisfies J2pX^qa Pi"ob{pX qa) £ {0, 1} for all p £ Q and X £ T. 

A pBPA is a pPDA with just one control state. Formally, a pBPA is understood as a 
triple A = (F, 6, Proh) where 5<ZV xV*. 

In the rest of this paper we adopt a more intuitive notation, writing pX ^ qa instead 
of Prob{pX — > qa) = x. A configuration of A is an element of Q x F*. The set of all 
configurations of A is denoted by C(A). We also assume (w.l.o.g.) that if qa £ 5, 

then |a| < 2. It is easy to transform an arbitrary pair (A, F), where A is a pPDA and F is a a 
PCTL formula or w-property, into another pair (A', F') such that A' satisfies the assumption 
above and A satisfies F if and only if A' satisfies F' . Moreover, the transformation takes 
linear time. For instance, a transition rule pX —> qYZW of A is transformed into two 

X 1 

transitions pX — > p'Y'W and p'Y' — > qYZ in A', where p', Y' are a fresh control state and 
a fresh stack symbol, respectively. 

To A we associate the probabilistic transition system Ta where C(A) is the set of states 
and the probabilistic transition relation is determined as follows: pX(3 qa(3 is a transition 
of 7a iff ^ qa IS a. transition of A and j3 £T* . 

The model checking problem for pPDA configurations and PCTL formulate (i.e., the 
question whether pa \='^ ip for given pa, cp, and u) is clearly undecidable for general valua- 
tions. Therefore, we restrict ourselves to regular valuations which to every a £ Ap assign a 
regular set of configurations: 

Definition 2.3. A /S.- automaton is a triple A = {St, 7, Acc) where St is a finite set of states 
s.t. Q <^ St, J : St xT ^ St is a (total) transition function, and Acc <^ St a set of accepting 
states. 

The function 7 is extended to the elements of F* in the standard way. Each A- 
automaton A determines a set C{A) C C(A) given by pa £ C{A) iff j{p,a^) £ Acc. 
Here a^ is the reverse of a, i.e., the word obtained by reading a from right to left. 

We say that a set C C C(A) is regular iff there is a A-automaton A such that C = C{A). 

In other words, regular sets of configurations are recognizable by finite-state automata 
which read the stack bottom-up (the bottom-up direction was chosen just for technical 
convenience) . 

An important technical step is that one can reduce the model-checking problem for reg- 
ular valuations to the problem for simple valuations that assign to each atomic proposition 
a simple set of configurations. Loosely speaking, a set of configurations is simple if we can 
decide whether a configuration belongs to the set by inspecting only its control state and 
its top stack symbol. 

Definition 2.4. A set of configurations C C C(A) is simple if there is a set G Qx (FU{e}) 
such that for each pa £ C(A) we have that pa £ C iff either a = e and pe £ G, or a = X(3 
and pX £ G. 

The reason why we only need to consider simple valuations is a bisimilarity property. 
Let Ci, • • • ,Ck ^ C(A) be regular sets of configurations, and assume that all we can observe 
from a configuration is whether it belongs to Ci for every \ < i < k. Loosely speaking, 
Lemma l2.5l below states that we can effectively construct another pPDA A' and simple sets 
of configurations C^, • • • ,C'^ C C(A) such that A and A' are bisimilar with respect to these 
observables (in the usual definition of bisimilarity one observes transitions between config- 
urations, while here we observe the configurations themselves, but otherwise the notion is 



6 



J. ESPARZA, A. KUCERA, AND R. MAYR 



the same) . The idea of the construction is to take A-automata Ai, - ■ • , At accepting the 
sets Ci, • • • and construct A' such that the following holds: If the current configuration 
of A is pa, then in the simulating configuration of A' the topmost stack symbol stores the 
states reached by the A-automata after reading from the initial state p. Although this 
construction is standard (see, e.g., |EKS03j ). we include an explicit proof for the sake of 
completeness. 

Lemma 2.5. For each pPDA A = (Q, T, 6, Prob) and regular sets Ci, • • • , C C(A) there 
effectively exists a pPDA A' = {Q,T' ,6' , Prob'), simple sets €[,■■■ ,C'f^ C C(A'), and an 
injective mapping Q : C(A) C(A') such that for each pa G C(A) the following conditions 
are satisfied: 

• for each 1 < j < k we have pa G Cj iff G (pa) £ C'^; 

• if pa qP, then G{pa) ^ Q{qf}); 

• ^/ G{pa) ^ s for some s G C(A'), then there is pa q(3 such that Q{q(3) = s. 
Moreover, if C CI C(A') is regular, then Q~^{C) is also regular. 

Proof. For each 1 < i < /c, let .Aj = {Sti,'ji, Acci) be a A-automaton such that C{Ai) = Ci. 
Let States = Y[i=i YipeQ "^^j- ^'^^ given s G States, I < i < k, and p G Q, we denote by 
s{i,p) the component of s which corresponds to i and p. 

We put r' = r X States. The transition function 6' and probabilities Prob' are defined 
as follows: 

• if pX ^ qe Ci 5, then p{X, s) ^ qe for each s G States; 

• if pX qY G 5, then p{X, s) ^ q{Y, s) for each s G States; 

• iipX qYZ G 5, thenp(X, s) —>■ q(Y,i){Z, s) for all s,t € States such that ')i{s{i, r), Z) = 
t{i, r) for all 1 < i < A; and r C Q. 

So, the A-automata ^i, • • • ,Ak are simulated "on-the-fiy" by storing the vector of current 
states directly in the stack. Hence, the information whether a given Ai accepts the current 
configuration is available in the topmost stack symbol. For every 1 < i <k, the underlying 
set Gi of C'^ (see Definition 12. 4|) is defined by 

G^ = {p{X, s) I -fi{s{i,p),X) G Acci} U {pe \ pe G Ci} 

The function Q is defined by Q{pe) = pe, and G{pXi ■ ■ ■ X}S) = p{Xi, si) ■ ■ ■ {X^, s^), where 
SkihQ) = and Sj{i,q) = ji{sj+i{i,q), Xj^i) for all I < j < k. It follows immediately 
from the definition of 6' and Prob' that the parts of 7a and 7a' which are reachable from 
pa and G{pa) are isomorphic (for every pa G C(A)). 

Let C C C(A') be a regular set of configurations. Since some configurations of C can 
be "inconsistent" in the sense that the vectors of states that are stored together with the 
original stack symbols do not correspond to a valid computation of the Ai automata, the 
set G~^{C) is not a simple projection of C "forgetting" the vectors of states from the stack 
symbols. Fortunately, Q{C{A)) is (obviously) a regular set, so we can construct a A'- 
automaton recognizing the set C H Q{C{A)) and apply the mentioned projection. □ 

3. Random Walks on pPDA Graphs 

In this section we address the following problem. Let A be a pPDA, let piai be an 
initial configuration, let Ci, C2 be two simple sets of configurations, and let p be a threshold 
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probability. Is the probability of executing a run piai; p2a2; pscts ■ ■ ■ that satisfies C1UC2, 
denoted by V{piai,Cil{ C2), at least p7 We show that the problem is decidable. 

The plan of the section is as follows. First, we show in Lemma [3.4l that V{piai,Ci U C2) 
is equal to a polynomial expression in the following probabilities: 

• Let pX be an initial configuration (notice that there is only one symbol on the stack), 
and let g be a control state q. The probability of reaching qe visiting only configurations 
of Ci \ C2 along the way is denoted by [pXq] 

• Let pX be an initial configuration and let r be a threshold probability. The probability 
of reaching some configuration of C2 with nonempty stack, visiting only configurations of 
Ci along the way, is denoted by [pX»] . 

Second, in Theorem 13.51 we show that the probabilities [pX(/] and are the least 

solution of a system of quadratic equations. So our original problem reduces to determining 
whether a polynomial expression on this least solution has at least the value p. Finally, we 
observe in Theorem 13 . 71 that this question can be reduced to deciding the truth of a formula 
in the first-order arithmetic of the reals (i.e., in the theory (R, +, *, <)). Since this theory 
is known to be decidable |Tar51j . our original question is decidable. 

For the rest of this section, let us fix a pPDA A = {Q,T,5, Prob) and two simple sets 
Ci,C2 C C(A). Let Gi,G2 ^ Q X (FU {e}) be the sets associated to Ci,C2 in the sense of 
Definition in 

Definition 3.1. To simplify our notation, we adopt the following conventions: 

• For each C C C(A), let C = C \ {Qx{e}). Observe that if C is simple, then so is C*. 

• For every C C C(A) and every /3 G F*, the symbol CP denotes the set {pal3 \ pa G C}. 

• For all p,q G Q and X G T, we use [pXq] to abbreviate V{pX,Ci\C2U {qs}), and [pX*] 
to abbreviate V{pX,CiUCl). 

• Let A be a set of finite paths which end in the same state t, and let B a set of finite or 
infinite paths that start in t. Then the symbol AQ B denotes the set of paths {u; w \ v G 
A,t;w £ B}. 

The proof of Lemma [3.4l our first milestone, requires the following two auxiliary results: 

Lemma 3.2. Let T = (£',—>, Pro6) be a probabilistic transition system. Let s,t £ S and 
Ci,C2 C S. Further, let A = FPath{s, {Ci\C2)ll{t}) and B = FPath{t,CiU €2). Then 

^ V{Run{w)) = ^ V{Run{w)) ■ ^ V{Run{w)). 

Proof. Immediate. □ 

Lemma 3.3. For all pa G C(A) and (3 £ T* we have that V{pa,CiU C2) is equal to 
V{pal3,C'i[iUC2l3). 

Proof. For every finite path w = piai; • • • \PnCX-n of FPath{pa), let w^^ denote the finite 
path piaif5; ■ ■ ■ ;pnanP of FPath{paf3). Realize that V{Run{w)) = V{Run{w~^^)), because 
w and w^^ execute the same transitions. One can easily verify that w E FPath{pa,CiU C2) 



8 



J. ESPARZA, A. KUCERA, AND R. MAYR 



iff w+l^ e FPath{paj3,Clj3UC2l3). From this we get 

V{pa,CiUC2) = V{Run{w)) 

w£FPath(pa,CiUC2) 

= V{Run{w)) 

w e FPath {pa0filf3UC2P) 

□ 

Now we show how to compute V{pXi ■ ■ ■ X„, Ci U C2) from the finite family of all [pXg], 
probabilities. First, realize that 

V{pX,---Xn,CiUC2) = [pXi.] + Y,[pXiq]-V{qX2---Xn,CillC2) 

q&Q 

The meaning of this equation is intuitively clear. If we repeatedly expand the probabilities 
of the form V{qXj ■ ■ ■ Xn,CiU C2) in the above equation (until j becomes n), we obtain the 
equation presented in the following lemma: 

Lemma 3.4. For each pXi ■ ■ • X„ G C(A) where n>0 we have that V{pXi ■ ■ ■ Xn,CiU C2) 
is equal to 

n i—1 n 

*=i (gi.-,9i)eQ' J=i (<?i, -,9n+i)eQ"+i i=i 

where p=qi where p=qi and qn+i£&C2 

with the convention that empty sum is equal to and empty product is equal to 1. 

Proof. By induction on n. For n = we have that V{p£,CiU C2) is equal either to 1 or 0, 
depending on whether pe belongs to C2 or not, resp. Now let n > 1, and let /3 denote the 
sequence X2 • • • The set Run{pXi(3, C1UC2) is equal to 

l+J Run{w) 
weFPath{pXi/3,CiU C2) 

Let C = {qaP \ q £ Q,a £ F+j. We have that 
FPath{pXil3,CiUC2) = FPath{pXi(3,CinC'UC2nC') W 

[+J FPath{pXi(3, {Ci\C2)r\C' U {qf3}) FPath{ql3,Cil^ C2) 

Now observe that for every simple set C C C(A) we have that C Pi C = C (3. Hence, the 
above equation can be rewritten as follows: 

FPath{pXif],CiUC2) = FPath{pXif3,C'iliUe^l3) W 

[+J FPath{pXi(3, {Ci\C2)' f3U {q/3}) Q FPath{q(3,Cil( C2) 

qeQ 

Using Lemma Eini and Lemma \'A.'2[ we obtain that 
V{j)Xi(3,CiUC2) =V{pXi,CiUCl) + 

Eqeo npXiP, (Ci \C2) U {q(3]) ■ r{qP, C1UC2) 
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This can also be written as 

V{pXi[j, Ci U C2) = [pXi*] + [pXiq] ■ V{qf3, Ci U C2) 

Now it suffices to apply induction hypothesis to 'P{qj3,CiU C2) and restructure the resulting 
expression. □ 

Now we show that the probabilities [pXt;] , form the least solution of an effectively 

constructible system of quadratic equations. This can be seen as a generalization of a similar 
result for finite-state systems |HJ94linY95j . In the finite-state case, the equations are linear 
and can be further modified so that they have a unique solution (which is then computable, 
e.g., by Gauss elimination). In the case of pPDA, the equations are not linear and cannot be 
generally solved by analytical methods. The question whether the equations can be further 
modified so that they have a unique solution is left open; we just note that the method used 
for finite-state systems is insufficient (this is demonstrated by Example 13. 6|) . 

Let V = {{pXq), {pX») \ p, q G Q, X G T} he a set of "variables". Let us consider the 
system of recursive equations constructed as follows: 

• if pX Gi\G2, then (pXq) = for each q £ Q; otherwise, we put 

{pXq) = ^ ^ • i'^^t) • + ^ ■ i^^l) + a; 

• if pX G G2, then {pX») = 1; if pX Gi U G2, then {pX») = 0; otherwise we put 

{pX») = 'Y X- ((ry.) + {rYt) • {tZ»)) + ^ x • {rY») 

pX^rYZ pX^rY 

The intuition behind these equations is easy to understand. For the sake of simplicity, 
assume Gi = Q x T and G2 = (this corresponds to Ci = C(A) and C2 = 0). In this case, 
we only have the two "long" equations. Consider the first one, the intuition for the second 
one being similar. In order to reach qe from pX, the pPDA must make at least one move. 
Since we assume than the transitions pX qa of a pPDA satisfy \a\ < 2, here are three 
possible kinds of moves: moves that increase the stack length by one, moves that do not 
change the stack length, and moves that decrease the stack length. The three summands in 
the equations correspond to these three kinds of moves. Since no transition can be executed 
when the stack is empty, the only way to reach qe by means of a length-decreasing move 
is to apply a transition pX A qe, if it exists (third summand). If the ffist transition is 
length-keeping, i.e., of the form pX rY, then, after the transition, we must reach qe 
from rY (second summand). Finally, if the ffi'st transition is of the form pX — > rYZ, then 
the pPDA must first go from rYZ to some configuration tZ along a path of configurations 
having with Z as bottom stack symbol, and then from tZ to qe. Intuitively (see the next 
theorem for the formal proof), the probability of reaching tZ from rYZ along such a path 
is equal to the probability of reaching te from rY, and so we get the ffist summand. 

For given t E [0, 1]' ^ ' , p, g G Q, and X G T we use {pXq)^ and {pX»)^ to denote the 
component of t which corresponds to the variable {pXq) and {pX») , respectively. The above 
defined system of equations determines a unique operator J" : [0, 1] I ^ I [0, 1] I ^ I where J^(t) 
is the tuple of values obtained by evaluating the right-hand sides of the equations where all 
(pXq) and {pX») are substituted with {pXq)^ and {pX»)^, respectively. 
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Theorem 3.5. The operator T has the least fixed-point fx. Moreover, for allp,q G Q and 
X £ T we have that (pXq)^ = \pXq] and (pX*)^^ = \pX»]. 

Proof. Since is monotonic and continuous, it has the least fixed point fj, = Vfc^o •^'^(0)' 
where is the tuple of zeros. One can readily check that the tuple vr of all [pXg] and 
probabilities forms a solution of the above system; this is done just by partitioning 
the associated sets of runs into appropriate disjoint subsets similarly as in the proof of 
Lemma 13.41 Hence, /i < vr. To prove that also tt < /x, we approximate the [p^g] and 
probabilities in the following way: For each k gNq we define 

• \pXqf = V{Run{w)) 

weFPath''{pX,Ci-^C2 U {qe}) 

• [pX»]^ = ^ V{Run{w)) 

w£FPath''{pX,CiUC^) 

Let TT*^ be the tuple of all \pXq\^ and [p-'^*]*^ probabilities. Clearly vr = limfc_^oo ^r*^ . By 
induction on k we prove that vr'^ < /i for each /c G No, hence also vr < as needed. 

The base case {k = 0) follows immediately. We show that if \pXq\^ < (pXq) ^ and 
[pXmf < ipXq)^, then also [pXq]^^^ < {pXq)^ and [pXmf+^ < {pXq)^. If dxGs, 
then \pXqf+^ = {pXq)^ = 0. Other wise, by applying the definitions we obtain 

[pXqf^^ = ^ X- ^ V{Run{vo)) 

pX^rYZ w£FPath''{rYZ,Ci^C2U{qe}) 

+ X] ^' X] V{Run{w)) 

pX^rY wGFPath''{rY,Ci\C2U{qe}) 

+ E ^ 

pX^qe 

and 

pX^rYZ *eQ pX^rY pX^qe 

Since 

V{Run{w)) = [rYqf, 

weFPath''{rY,Ci \C2 W {ge}) 

we have 

Y V{Run{w)) < {rYq)^ 

weFPath''{rY,Ci \C2 U {qe}) 

by induction hypothesis. Further, 

^ X- Y V{Run{w)) 

pX^rYZ w&FPath''{rYZ,Ci^C2U{qe}) 

is surely bounded by 

Y x-Y[rytf-[tZq]\ 

pX^rYZ 
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which is bounded by 

by induction hypothesis. To sum up, we have that [pXq]'^'^^ < (pXq)^. The inequahty 
[pX*]'^'^^ < {pX») ^ is proved similarly. □ 

Example 3.6. Let us consider the pBPA system A of Fig. ^ and let Ci = F*, C2 = {Z}. 
Then we obtain the following system of equations (since A has only one control state p, we 
write {X,») and {X,e) instead of {pX») and (pXp), resp.): 

{Z,.) = 1 

{Z,e) = x{I,e){Z,e) + {l-x){D,e){Z,e) 

(/,.) = + (/,£)(/,.)) 

(/,e) = x{I,e){I,e) + 1-x 

{D,.) = il-x)i{D,.) + {D,e){D,.)) 

{D,e) = {l-x){D,e){D,e) + x 

As the least solution we obtain the probabilities [Z,»] = 1, [Z,e] = 0, [/, •] = 0, [/, e] = 
min{l, (1— [D,»] =0, [D,£] = min{l, — x)}. Bv applving Lemma 13.41 we further 
obtain that, e.g., V{IIZ,CiUC2) = [I, •] + [!, e]- {[I, •] + [!, e]-[Z,»]) = min{l, {l-xf/x^}. 
□ 

In Example 13.61 it is possible to compute a closed form for the least solution of the 
system of equations, but in general this is not true. However, many important properties 
of the least solution are decidable, because the decision problem can be reduced to the 
problem of deciding the truth of a formula in the first-order theory of the reals. For our 
purposes, it suffices to consider the class of properties defined in the next theorem. 

Theorem 3.7. Let Const = Q U {[pXq], \pX»] \ p,q e Q and X G F}, where Q is the set 
of all rational constants. Let Ei,E2 be expressions built over Const using '■' and and 
/ei ~ S {<, =}. It is decidable whether Ei ~ £'2- 

Proof. We show that, due to Theorem 13.51 Ei ~ E2 is effectively expressible as a closed 
formula of (M, +,*,<). Hence, the theorem follows from the decidability of first-order 
arithmetic of reals |Tar51j . 

For all p,q G Q and X £ F, let x{pXq), x{pX»), y{pXq), and y{pX») be first order 
variables, and let X and Y be the vectors of all x{pXq), x{pX»), and y{pXq), y{pX») 
variables, respectively. Let us consider the formula $ constructed as follows: 

3X : 0<X <1 A X = r[X) 

A (vy : (0 < y < 1 A y = r{Y)) =^ x <y)) 

A Ei[X/7r] ~ E2[X/tt] 

Observe that the conditions X = J-{X) and Y = J-^Y) are expressible only using multipli- 
cation, summation, and equality. The expressions Ei[X/7r] and E2[X/Tr] are obtained from 
El and E2 by substituting all \pXq] and [pX»] with x{pXq) and x{pX»), respectively. It 
follows immediately that Ei E2 iS ^ holds. □ 
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Input: pX e C(A), < A < 1 
Output: V" 

1: := 0; 1; 

2 : for 2 = 1 to [- log2 A] 

3: if [pX.] + j:^^^^JpXq]>ir^^V')/2 

4: then V''' ~V'^)/2 

5: else F" := -V'^)/2 

6: fi 

Figure 2: Computing 

An immediate consequence of Theorem 13.71 is the following: 

Theorem 3.8. Let pa G C(A), ^ G Qn [0, 1], ~ G {<,<,>,>} and < A < 1. is 
decidable whether V{pa,Cih( C2) ^ Q- Moreover, there effectively exist rational numbers 
V^,^ such that < V{pa,CiUC2) < V and -V^ < A. 

Proof. We can assume w.l.o.g. that a = X for some X G T. Note that V{pX, C\ U C2) ~ iff 
[pXm] + 'Yliq£&C2 \P-^l\ ~ by Lemma IH3I Hence, we can apply Theorem 13. 71 The numbers 
are computable, e.g., by the algorithm of Fig. [5J □ 

4. Model Checking PCTL for pPDAs 

In this section we study the model-checking problem for PCTL formulas with regular 
valuations and pPDA. 

4.1. Qualitative Fragment of PCTL. We give a model checking algorithm for the qual- 
itative fragment of PCTL, i.e., for the fragment in which only and 1 are allowed as 
probability thresholds. 

Recall that in order to check if a CTL formula ip holds of a finite state system we 
first recursively compute the sets of states that satisfy the subformulas of (p lying right 
below in the syntax tree, and then we apply a semantic operator that gets these sets 
of states as inputs and produces the set of states satisfying ip as output. In the case of a 
PDA (no probabilities), these sets of states (they are now sets of configurations) can be 
infinite. Therefore, in order to apply a similar algorithm it is necessary to prove that the 
sets have a finite representation. This was done in |BEM97j : It was shown that in the case 
of regular valuations the sets are always regular, and so can be finitely represented by, say, 
finite automata. In this section we prove that the same property also holds for pPDA and 
for the qualitative fragment of PCTL, and that the constructions showing the regularity of 
the sets are effective. 

By Lemma l'2.5l we only need to show that if the sets of configurations satisfying the 
subformulas of ip are simple, then the set of configurations satisfying ip is regular. We need 
to consider four cases, corresponding to formulas of the form X^^if, X^^ip, piU ^^ip2, and 
ipilA^^ip2- they are dealt with in Lemma l4. 11 Lemma 14.21 and Lemma 14.31 

For the rest of this section we fix a pPDA A = [Q, F, 5, Proh). 

Lemma 4.1. Let C C C(A) be a simple set. The sets {pa G C(A) | V{pa, XC) = 1} and 
{pa G C(A) I V{pa,XC) = 0} are effectively regular. 

Proof. Follows immediate from the fact that pa has only finitely many successors in the 
probabilistic transition system associated to A.. □ 
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Lemma 4.2. Let Ci,C2 C C(A) be simple sets. The set {pa G C{A) \ 'P{pa,CiUC2) = 1} 
is effectively regular. 

Proof. Let R{pX) = {q ^ Q \ [pXq] > 0} for all p G Q, X G T. For each i G Nq we define 
the set Si C C(A) inductively as follows: 

• 5o = {qe I qe G C2} U {qXa \ [qX»] = 1, a G L*} 

. 5,+i = I [pX.] + EgeBipX) [p^Q] = 1 and G : qp G 5,} 

Using Lemma 1231 we can easily check that (J£o ~ ^ C(^) I 'P{pa.,CiU C2) = 1}. 
To see that the set IJi^o effectively regular, for each p (z Q we construct a finite 
automaton Aip such that L{^Ap) = {a £ T* \ pa £ Ui^o "^O- ^ A-automaton A recognizing 
the set IJ£o '^i ^^n then be constructed using standard algorithms of automata theory (in 
particular, note that regular languages are effectively closed under reverse). The states of 
A4p are all subsets of Q, {p} is the initial state, F is the input alphabet, the final states 
are those T ^ Q where for every q £ T we have that qe G C2 (in particular, note that is 

a final state), and the transition function is given by T — > [/ iff for every q £ T we have 

that [qX»] + Y.reR{qX)[l^^] = 1 and C/ = [Jg^^R{qX). Note that 0^0 for each X £T. 
The definition of A4p is effective due to Theorem 13.71 It is straightforward to check that 
L{Mp) = {a£r*\pa£ U£o Si}- □ 

Lemma 4.3. Let Ci,C2 C C(A) be simple sets. The set {pa £ C{A) \ V{pa,CiUC2) = 0} 
is effectively regular. 

Proof Let R{pX) = {q £ Q \ [pXq] > 0} for all p £ Q, X £ F. For each i G Nq we define 
the set Si C C(A) inductively as follows: 

• 5*0 = {qe I qe C2} 

• Si+i = {pXP I [pX»] = and Vg G R{pX) : qP £ Si} 

The fact Ui^o '^^ ~ {P^ ^ ^i'^) I T^ipo^i ^ C2) = 0} follows immediately from Lemma|H3I 
The set IJ^q '^i effectively regular, which can be shown by constructing a finite automaton 
Mp recognizing the set {a G F* | pa £ IJiZo^i}- This construction and the rest of the 
argument are very similar to the ones of the proof of Lemma 14.21 Therefore, they are not 
given explicitly. □ 

Theorem 4.4. Let (p be a qualitative PCTL formula and u a regular valuation. The set 
{pa £ C(A) I pa \='^ if} is effectively regular. 

Proof. By induction on the structure of (f. The cases when if = tt and ip = a follow 
immediately. For Boolean connectives we use the fact that regular sets are closed under 
complement and intersection. The other cases are covered by Lemma 14.11 14.21 and 14.31 
Here we also need Lemma 12. 51 because the regular sets of configurations must effectively be 
replaced with simple ones before applying Lemma l4.ll 14.21 and 14.31 □ 

4.2. Model Checking PCTL for pBPA Processes. In this section we consider arbitrary 
PCTL properties with regular valuations, but restrict ourselves to pBPA processes. We 
provide an error-tolerant model-checking algorithm. Since it is not so obvious what is meant 
by error tolerance in the context of PCTL model checking, this notion is defined formally. 
More precisely, we first show that for every formula there is an equivalent negation-free 
formula, and then we provide a definition for negation- free formulas. 
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Let T = (5, — >, Pro6) be a probabilistic transition system and < A < 1, let be a 
PCTL formula, and let be a regular valuation (i.e., for every atomic proposition a the 
set v{a) of configurations is regular). We observe that there is a negation-free formula (p' 
and a regular valuation u' such that fipY = Iv'Y ■ Fhst, negations can be "pushed inside" 
to atomic propositions using dual connectives (note that, e.g., -'{ipU-^ip) is equivalent to 
(pU^^tp). Moreover, since regular sets are closed under complement, [-■a]'^ is also regular 
for every a. We construct (p' by replacing each negation -la by a fresh atomic proposition 
b, and we extend to z^' by defining ^{b) = {-^aY . 

For every negation-free PCTL formula (p and valuation v we define the denotation of 
ip over T w.r.t. u with error tolerance A, denoted l^p^x, in the same way as {p}'^ ■ The only 
exception is p^i U ^^p2 where 

• if ~ G {<, <}, then y^U ~(^p2}'i = {s e S \ V{s, lipifxU Mx) ~ ^> + A} 

• if - G {>, >}, then l^, U ^^2^ = G | V{s, l^ifxU lp2\l) ^ Q - X] 

Notice that every negation- free formula ip satisfies \ipY ^ [vIa- 

An error tolerant PCTL model checking algorithm is an algorithm which, for each PCTL 
formula 99, valuation v, s £ S, and < A < 1, outputs YES/NO so that 

• if s G , then the answer is YES; 

• if the answer is YES, then s G [93]^. 

For the rest of this section, let us fix a pBPA A = (F, 6, Prob). Since A has just one (or 
"none") control state p, we write [X,»] and [X,£] instead of \pX»] and [pXp], respectively. 
We need the following obvious generalization of Lemma l4. II fuse the same proof): 

Lemma 4.5. Let C C C(A) be a simple set, g G [0,1], and G {<,<,>,>}■ The set 
{a G C(A) I V{a,XC) ^ g} is effectively regular. 

Proof. Immediate. □ 

The following lemma presents the crucial part of the algorithm. This is the place where 
we need the assumption that A is a pBPA. 

Lemma 4.6. Let Ci,C2 ^ C(A) be simple sets. For all g G [0,1] and < A < 1 there 
effectively exist A-automata A- and A- such that for all a G C(A) we have that 

• ifV{a,CiUC2) > g (or V{a, C1UC2) < g), then a G C{A^) (or a £ C{A-), respectively.) 

• if a e C{A^) (or a G C{A^)), then V{a,CiUC2) > g - X (or V{a,CiUC2) < g + \, 
respectively.) 

Proof. We describe just the construction of A- (the A-automaton A- is constructed simi- 
larly). Let 5 = {X e F I [X,e\ / 1}. For each /3 G 5* we define the set «(/?) = {a G F* | 
ot\s = /^li where a\s is the word obtained by deleting in a all occurrences of symbols in 
F \ 5". It follows directly from Lemma 13.41 that for all (3 £ S* and a G Cl{(3) we have that 
V{(5,CiUC2) = V{a,CiUC2). Further, for ah n G No and /3 G ULo -S*' we define the set 

J a(/3) if a G 5^ A i < n 

|{aa' I a G a(/3),a' G F*} if a G 5" 

We prove that for every < A < 1 there effectively exist n G No and Q C IJ^q 5* such that 
for every a G F* we have that 

• if V{a, Ci U C2) > g, then a G U/Jeg Genn{(3); 



GenniP) 
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Input: pBPA A, < A < 1 

Output: n, K, u, [X,»Y, [X,eY, [X,.]^, [X,e]^ 

1: S:={X er\[X,e]^l}- 
u := 1; n := oo; 
for each X G S" do 

[X,eY :=0; [X,.]' := 0; [X^ef := 1; [X, := 1; 
done 
repeat 

for each X G F do 

avg- :={[X,er-[X,eY)/2; 
avg* := ([X,.]"-[X,.]0/2; 



2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 



if [X, e] > avg'^ then [X, e] := avg^; 

else [X, e]" := avg^; 
if [X,«]>avg' then [X,»Y := avg'; 

else [X, := avg*; 

done 

u := u/2] 

K := max{[X,e]" \ X £ S}; 
if K < 1 then n := [(log(A/3)/ log k] 
until K<1 and n(z^ + z^(n + 1)(1 + z^)") < A/3 

Figure 3: A part of the algorithm for pBPA 



• if a G [jpeg Genn{(3), then V{a, Ci U C2) > Q — X- 

This suffices for our purposes, because the set U/jee Genn{l3) is clearly recognizable by an 
effectively constructible A-automaton A^. 

The crucial part of the algorithm for computing the set Q is shown in Fig. |31 The 
algorithm starts by computing the set 5 (note that 5 is effectively computable due to 
Theorem 13 .71) . For each X £ S, there are four rational variables [X, e]^, [X, e]", [X, "J^, 
and [X, whose values are lower and upper approximations of the probabilities [X, e] and 
[X, •], resp. These variables are initialized in lines 3-5 and successively refined in lines 
7~14. Note that the conditions of the if statements in lines 10 and 12 are effective due to 
Theorem 13. 71 The current "precision", i.e., the difference between the upper and the lower 
approximation is stored in the rational variable v. The subtle point is the termination 
condition. First, one necessary condition for termination is that k = max{[X, e]" | X G S"} 
becomes less than one. This must happen eventually, because [X, e] < 1 for every X G S". 
An important observation is that k can only decrease by performing the assignment in 
line 16. This means that n = [log (A/3)/ log k] also only decreases (since both A and k 
are less than 1, we have log(A/3)/ log k = | log(A/3)|/| log k|; and if < k' < k < 1, 
then |logK'| > |logK|). Therefore, we eventually find a sufficiently small v such that 
n{v + z/(n + 1)(1 + vT) < A/3. 

The output of the algorithm of Fig. |21are the (values of the) variables n, k, [X, e]^, 
[X,e]", [X,.]^ and [X,.]" where X ranges over S. For each (3 G S*, let V\l3,CiUC2) and 
V^{(3,CiU C2) be the lower and upper approximations of V{(3,CiLl C2) obtained by using 
the formula of Lemma [3.41 where [X, e]^, [X, "J^, and [X, e]", [X, are used instead of 
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[X, e], [X, •], respectively. The set Q is constructed as follows: 

g = {(3£S'\0<i<n,V''{f3,Cil^C2)>Q} 
U {/3g5"|P"(/3,CiZ^C2)>^-A/3} 

To verify that the set G has the properties mentioned above, we need to formulate two 
auxiliary observations. 

(a) for all /3 G S*" and a G F* we have that 

\V{P,Cil^C2)-V{Pa,Cii^C2)\ <A/3 
This follows immediately from the following (in)equalities: 

V{f3a,CiUC2) = V{f3,CiUC'2) + V{(3,Ci\C2U{e])-V{a,CiUC2) 
V{(5,CiUC2) < V{(3,CiUCl) + V{f5,Ci^C2U{e}) 

V{(i,Ci\C2U{e}) < A/3 

The first two (in) equalities are obtained just by applying Lemma 13.41 The last one is 
derived as follows: 'P(/3, Ci\C2^ {e}) is surely bounded by k" (by Lemma f3.4l and the 
definition of k). Since n = [log(A/3)/log k] , we have n • logK < log(A/3). Hence, 
logK" < log(A/3), thus < A/3. 

(b) for each (3 £ Ur=o have that 

P"(/3, Ci U C2) - Vi(3, Ci U C2) < A/3 

Let k = length{P). A straightforward induction on k reveals that V'^{(3,CiU C2) < {k + 
1) • (1 + uf. Now we prove (again by induction on k) that 

V''{l3,CiUC2)-V{l3,CiUC2) < k{u + u{k + l){l + u)^) 

The base case (when A; = 0) is immediate, because 'P'^{e^CiU C2) = 'P{e,CiU C2). Now 
let (3 = Xp'. By definition, P"(X/3', Ci C2) - V{Xp',Ci^C2) is equal to 

[X,.]^ + [X,er-P^{p',CiUC2) - {[X,.] + [X,£]-V{(3',CiUC2)) (4.1) 

Since [X, < [X, •] + u and [X, e]" < [X, e] + u, the expression H4.1() is bounded by 

1/ + [X,£]-{V''{l3',Cil(C2)-V{f3',CiUC2)) + u-V''{(3',CiUC2) (4.2) 

By applying induction hypothesis and the facts that [X,e\ < 1 and V^{P,Cih( C2) < 
(/c + 1) • (1 + v)^ (see above), we obtain that the expression ()4.2() is bounded by 

iy + k{u + u{k + 1)(1 + uf) + v{k + 1)(1 + vf 

which is bounded by {k + l){v + v{k + 2){l + v)^^^) as required. This finishes the inductive 
step. 

Since n{v + v{n + + v^) < A/3 and A; < n, we have ViP^i^ C2) - V{P,Cii^ C2) < 
k{u + iy{k + l){l + u)'') < A/3. 

Now we are ready to prove that the set Q has the required properties. Let a £ T* such that 
Vi^ajCiU C2) > Q, and let (3 = a\s- There are two possibilities: 

• length{(3) < n. Then V^{(3^CiU C2) > Q, hence P £ G and a G U/Jee G'en„(/3). 

• length{0) > n. Let /3 = 77' where length{'j) = n. Due to the observation (a) above we 
have that 7^(7, Ci Z//C2) > g — \/3, hence also 'P"(7,Ci WC2) > q — X/3, which means that 
7 G ^ and thus a G UsgG GeUniP)- 
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Now let a G G'en„(/3) for some P & G. Again, we distinguish two possibilities: 

• length{(3) < n. Then P"(/3, Ci C2) > £», which means that V [(5 ,CxU Ci) > £> - A/3 by 
the observation (b) above. Hence, V{a^C\lAC2) > g — A/3. 

• lengthiP) = n. Then P"(/3,Ci > £»-A/3, which means that P(/3,CiZ^C2) > £<-2A/3 
due to the observation (b). Further, for every a' G T we have that V{f3a' ,CiU C2) > g — X 
due to the observation (a) above. Hence, V{a,CiU C2) > g — \ as required. 

The automaton A- is constructed similarly. Here, the set Q is computed using the 
lower approximations [-'^i and [X, e]^. Since this construction is analogous to the one 
just presented, it is not given explicitly. □ 

Theorem 4.7. There is an error-tolerant PCTL model checking algorithm for pBPA pro- 
cesses. 

Proof. The proof is similar to the one of Theorem 14.41 using Lemma 14.51 and 14.61 instead of 
Lemma 14.11 14.21 and 14.31 Note that Lemma 12.51 is applicable also to pBPA (the system A' 
constructed in Lemma 12.51 has the same set of control states as the original system A). □ 

5. Model Checking w-regular Specifications 

In this section we show that the qualitative and quantitative model-checking problem 
for pPDA and w-regular properties are decidable. At the very core of our result are obser- 
vations leading to the definition of a finite Markov chain Ma- Intuitively, each transition 
of Ma corresponds to a sequence of transitions of the probabilistic transition system 7a 
associated to A. This allows to reduce the model-checking problem to a problem about 
Ma, which, since Ma is finite, can be solved using well-known techniques. In |EKM04] . the 
Markov chain Ma was used to show that the qualitative and quantitative model-checking 
problem for properties expressible by deterministic Biichi automata is decidable. Later, 
it was observed in |BKSn5j that the technique can easily be generalized to deterministic 
Muller automata. Thus, the decidability result was extended to all w-regular properties. 
In this paper we go a bit further, and prove the decidability of a slightly larger class. The 
previous result about the w-regular case follows as a corollary. 

The section is structured as follows. Given a pPDA A, we first introduce the notion of 
minima of a run and A-observing automaton. We use observing automata as specifications: 
an infinite run satisfies the specification iff it is accepted by the automaton (section 15. 1|) . 
Using the notion of minima, we define the finite Markov chain Ma (section 15. 2j) . and 
show that the probability that a run is accepted by a A-observing automaton is effectively 
expressible in (M, -|-,*,<) f section 15.3(1 . Finally, we show that the model-checking problem 
for w-regular properties is a special case of the problem of deciding if a run is accepted by 
a A-observing automaton with at least a given probability (section l5.4() . 

For the rest of this section, we fix a pPDA A = {Q, P, 6, Proh). 

5.1. Minima of a run. Loosely speaking, a configuration of a run is a minimum if all 
configurations placed after it in the run have the same or larger stack length. 

Definition 5.1. Let w = PiCii',P20(2, • • ■ be an infinite run in 7a. A configuration pjOj is a 
minimum of w if \ai\ < \aj\ for every j > i. We say that pjQj is the A;*'* minimum of w if 
PiUi is a minimum and there are exactly k — 1 indices j < i such that PjCtj is a minimum. 
We denote the k''^ minimum of w by mmf^(w). 
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Sometimes we abuse language and use mini(u;) to denote not only a configuration, but the 
particular occurrence of the configuration that corresponds to the i^^ minimum. 

Example 5.2. In the run wi = {Z; DZ)'^ of the pBPA shown in the introduction we have 
minj(i(;i) = Z for every i > 1. In the run W2 = Z]DZ;DDZ] ... we have VD.m.i{w2) = Z 
and m.va.i{w2) = D for every i >2. Every odd configuration of wi is a minimum, and every 
configuration of W2 is a minimum. □ 

Since stack lengths are bounded from below, every infinite run has infinitely many 
minima, and so it can be divided into an infinite sequence of fragments, or "jumps", each 
of them leading from one minimum to the next. 

We are interested in those properties of a run that can be decided by extracting a 
finite amount of information from each jump, independently of its length. Consider for 
instance the property "the control state p is visited infinitely often along the run" . It can 
be reformulated as "there are infinitely many jumps along which the state p is visited". 
In order to decide the property all we need is a bit of information for each jump, telling 
whether it is "visiting" or "non- visiting" . We consider properties in which this finite amount 
of information can be extracted by letting a finite automaton go over the jump reading the 
heads of the configurations: 

Definition 5.3. Given a configuration pXa of A, we call pX the head and a the tail of 
pXa. The set Q x T of all heads of A is also denoted by 7i(A). 

More precisely, we consider automata with the set of heads as alphabet. An oracle tells 
the automaton to start reading heads immediately after the run leaves a minimum (i.e., the 
first head read is the one of the configuration immediately following the minimum), stop 
after reading the head of the next minimum, report its state, and reset itself to an initial 
state that depends on the head of the minimum. 

Definition 5.4. A A-observing automaton is a tuple A = {A,£^,ao, Acc) where A is finite 
set of observing states, ,^ : ^ x 'H(A) — > A is a (total) transition function, oq £ A is an 
initial state, and Acc is a set of subsets of A, also called an acceptance set. 

Let w be an infinite run in 7a and let z S N. The i^^ observation of A over vu, denoted 
Obsi{w), is the state reached by A after reading the heads of all configurations between 
va\n.i{w) and vaiD.ij^i{w) , including vavciij^iiw) but not including m.va.i{w). ^ The observation 
of A on w, denoted by Obs{w), is the sequence 0bsi{w)0bs2{w) . . .. 

We say that an infinite run w G Run{pX) is accepting if the set of states of A that 
occur infinitely often in Obs{w) belongs to Acc; otherwise, w is rejecting. 

Example 5.5. Figure |1] shows a A-observing automaton for the pBPA of the introduction 
(see also Figure ^). For every infinite run w and every z > 0, we have Obsi{w) = 6 if some 
configuration of the i^^ jump has Z as topmost stack symbol. So a run is accepting iff it 
visits configurations with head Z infinitely often. □ 

For the rest of the section we fix a A-observing automaton A = {A,^,aQ, Acc). Let 
Run{pX, Acc) be the set of all accepting runs initiated in pX. Our aim is to show that 
V{Run{pX, Acc)) is effectively definable in (M, + ,*,<). 



'Notice that the automaton starts observing after the first minimum of the run. 
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I,D Z,I,D 




Acc={{ai}} 



Figure 4: An observing automaton 



5.2. The Markov chain Ma- For all pX G W(A) and all i G N we define a random 
variable V^^^ over Run{pX). Loosely speaking, V^^^ assigns to a run starting at the con- 
figuration pX the head of its i*^ minimum, and the i^^ observation of the A-observing 
automaton A. Formally, the possible values of V^^ are pairs of the form {qY,a), where 
qY G 'H(A) and a £ A. There is also a special value _L, where _L W(A) x A. For a given 
w G Run{pX), the value Vpxi'^) is determined as follows: If w is finite, then Vp^^{w) = _L; 

otherwise, V^x(^) ~ (9^' Obsi{w)), where qY is the head of minj(7i;). Notice that the 
random variables are well defined, because they assign to each run exactly one value. 

Given possible values vi, . . . ,Vn for the variables , . . . , V^"^ , we are going to prove 
the following two results: 

• the probability that a run satisfies V^^ = vi, . . . , = Vn is expressible in (R, +, *, <) 
(Lemma I5.1()|) : and 

(i+l) (i) 

• the probability that V^^ = fj+i depends only on the value of V^^, but neither on i nor 

on the value of V^^ for k < i (Lemma 15.111 and I5.12() . 

The second result will allow us to define the finite Markov chain Ma, while the first one 
will show that its transition probabilities are expressible in (M, +, *, <). 

The proof of Lemma l5.1Ul is rather technical (as we shall, see. Lemma 15.111 and l5.12l are 
easy corollaries of Lemma l5.1U() . We need three auxiliary lemmas. Intuitively, the first one 
states that the probability of executing an infinite run from a configuration pX is equal to 
the probability of executing an infinite run from pXP such that the stack content never goes 
"below" j3. For every finite or infinite path w = piai;P202; • • • in Ta and every j3 G F*, the 
symbol w~^^ denotes the path piai/3;p2«2/3; ■ ■ • obtained from w by concatenating (3 to the 
stack content in every configuration. Similarly, if i? is a set of paths in 7a and /3 G F*, then 
[R]+P denotes the set \ w£ R}. 

Lemma 5.6. Let pX G QxF and (3 G F*. Then V{[IRun{pX)\^'^) = V{IRun{pX)). 
Proof. Let Dead = Qxje} U {qYa \ qY has no transitions in 5, a G F*}. We have that 
V{{IRun{j)X)]+^) = 1 - V{pXf3,C{Ay/3 U Dead(3) 

= 1 - V{pX,C{J\) U Dead) (by Lemma ESI 

= V{IRun{pX)). 

□ 

The second lemma states that prefixing a measurable set of runs with a finite path 
yields a measurable set of runs, and relates the probabilities of both sets. 

Lemma 5.7. Let sq; • • • ; Sn be a path in a probabilistic transition system, and let R be a 
measurable subset of Run{sn)- Then {sq; • • • ; Sn} Q R is a measurable subset of Run{sQ), 



20 



J. ESPARZA, A. KUCERA, AND R. MAYR 



and moreover V{{so; ■ ■ ■ ; s„} Q R) = Tlf^^Xi ■ V{R), where Si Sj+i for every < i < n. 
( The '0 ' operator has been introduced in Definition I'j.ll ) 

Proof. Standard. □ 

The third lemma shows that the probabihty of starting from the configuration qY 
reaching the configuration qe with the observing automaton in state a is expressible in 
(M, +,*,<). 

Definition 5.8. Let R he a P-measurable set of runs of Ta starting at the same initial 
configuration. We say that V{R) is well-definable if there effectively exist a pPDA A' and 
a finite family of probabilities of the form V{Run{qY,Ci U C2)), where qY € TC{A') and 
Ci,C2 ^ C(A') are simple sets, such that V{R) is effectively definable from this family of 
probabilities using only summation, multiplication, and rational constants. 

Note that if P(-R) is well-definable, it can be expressed in (M, +, *, <) using the results of 
Section |31 

For all qY G W(A), r G Q, Z G T, and a G ^, let Run{qY,r, Z,a) C Run{qY) be the 
set of all runs w = sq; ■ ■ ■ ; Sn such that sq = qY, Sn = re, and the automaton A reaches 
the state a after reading the heads of configurations sq, • • • , Sn-i,rZ. 

Lemma 5.9. V{Run{qY,r, Z,a)) is well-definable. 

Proof. We put A' = {QxA,T,5' ,Prob') to be the synchronized product of A and A, i.e., 
{p,a)X ^ {t,a)a is a rule of A' iff pX ta is a rule of A and ^{a,pX) = a. Let 
A = {a G A \ (^{a, rZ) = a}. Now we can easily check that V{Run(qY, r, Z, a)) is equal to 

V{{q,ao)Y,C{A')U {{r,a)e}) 

□ 

We can now prove our main technical result: 

Lemma 5.10. For all pX G W(A), n G N, and vi,--- ,Vn & {Ti{A)xA) U {_L}, the proba- 
bihty of V^^j=vi A ■■■ AV^^^ =Vn is well- definable. In particular, for every rational constant 
y there is an effectively constructive formula of (M, +,*,<) which holds if and only if 

Proof. By induction on n we prove that "Pl^x*"^! ^' ' ' ^ ^pX~'^n) is well-definable. The 
base case when n = 1 follows immediately, because 'Pi^px—'^i) equals either V{IRun{pX)), 
1 — T'{IRun{pX)), or 0, depending on whether vi = {pX,ao), vi = _L, or {pX,ao) ^ v\ ^ 
_L, respectively. Observe that V{IRun{pX)) = 1 — V{pX,C{A) U Dead), where Dead = 
Qxje} U {qYa \ qY has no transitions in 5, a G F*}. 

Now let n > 2. For each 1 < i < n, let Sati be the set of all runs that satisfy 

n-i) = 0, which is decidable by induction hypothesis, then 
V{Satn) = as well. If V{Satn~i) 7^ and there is an i < n — 1 such that Vi = _L, then for 
all j < n — 1 we have that Vj = _L, and V{Satn) is equal either to V{Satn~i) or 0, depending 
on whether u„ = _L or not, respectively. If V{Satn-i) / 0, / _L for all i < n — 1, and 
Vn = -L, then V{Satn) = 0. So, the only interesting case is when T'{Satn-i) / and Vi -L 
for alH < n. Since 

ViV^X^=Vn I Satn-l) 



V{Satn) 



V{Satn- 
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and V{Satn-i) is well-definable by induction hypothesis, it suffices to show that the con- 
ditional probability T^iVpx —^n \ Satn~i) is also well-definable. For this we use a general 
result of basic probability theory saying that if A, B are events and B = [tli^jBi, where / is 
a finite or countably infinite index set, then 

E^<,InA\B,).V{B,) 



V{A I B) 



V{B) 



An immediate consequence of this equation is that if the probability V{A\Bi) is independent 
of i, then V{A\B) = V{A\Bi). In our case, A is the event V^^=Vn, and B is Satn-i- Let 

Chop = {w{0); • • • ; w^mmn-iiw)) \ w G Satn-i}- 

Observe that if y G Chop, then the last configuration of y is of the form pn-iXn-ia. We 
denote the a by Stack (y). For every y G Chop, let 

Sat^^iiy) = {y} [/i?^.n(p„_iX„_i)]+^*-*=(j') (5.1) 

Now we can easily check that 

Satn^i = 1+J Satn^i{y) 

y€ Chop 

Hence, Chop plays the role of I, and Satn-i{y) plays the role of Bi. We show that 
'P{V^'^ =Vn I Satn-i{y)) is independent of y, which means that 

V{V^;$=Vn I Satn-l{y)) = V{V^'$=Vn \ Satn^i). 

By definition of conditional probability. 



V{V^^ =v^ I Sat^.,{y)) = T,^SaU^_^{y)) 

The denominator of the fraction in equation (|5.2j) is well-definable, because 

V{Satn-^i{y)) = V{Run{y)) ■ V{IRun{pn-iXn-i)) 
Here we used Lemma EH Lemma 1^771 and equation (|5.1() . Now we show that 'PiVpx —'^n A 

Satn-i{y)) is also well-definable. Let R be the set of all runs satisfying =Vn f\ Satn-i{y) , 
and let = {pnXn,<in) and a = Stack{y). Obviously, each w £ R starts with y. Now let 
us consider what transitions can be performed from the final state pn-iXn^ia of y. 

• Obviously, transitions which decrease the stack cannot be performed, because p^-i^n-iCK 
would not be a minimum then (i.e., w would not belong to R). 

• If a transition of the form pn~iXn-ia ^ rZa is performed, then rZa must be the n-th 
minimum, because the stack cannot be decreased below Z (otherwise, pn-iXn-ia would 
not be a minimum). So, \i w £ R, we must have that rZ = PnXn and ^(ao, rZ) = an- 

• If a transition of the form p.„_iX„_ia ^ rPQa is performed, then the stack cannot be 
decreased below Q. Now there are two possibilities: 

— If the stack is never decreased below P, then the configuration rPQa is the n-th mini- 
mum. Hence, if w £ R, we must have that rP = PnXn and ^(ao, rP) = an- 
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— If the stack is decreased below P, i.e., if a sequence of transitions is performed of 
the form rPQa tQa (where the stack is never decreased to Qa except in the last 
configuration), then tQa is the n-th minimum. Hence, ii w & R, we must have that 
tQ = PnXn and the automaton A reaches a„ by reading the word consisting of heads 
of configurations in the sequence rPQa — >* tQa. 

From the above discussion, it follows that R can be partitioned as follows: 

R = l+J {y}Q{pn-iXn-ia;,pnXna}Q[IRun{pnXn)]^" 

l+J {y] {Pn-lXn-ia]PnXnYa] [IRun{pnXn)]^^'' 

P-n-lXn-l^PnXnY 

YeT 

l+J {y} [Run{qY,pn,Xn,anT'' [/i?7/n(p„X„)]+" 

qeQ,Yer 

Using Lemma l5.6( Lemma 15.71 Lemma 15.91 and the above equation, we obtain that 
'P{VpX=^n A Satn-i{y)) = V{Run{y)) ■ V {IRun{pnXn)) ■ S 



where 



S = X] ^ + 

Pn-lXn-l^PnXn Pn-lX„-l^PnX„Y 

yer 

^ X ■V{Run{qY,pn,Xn,an)) (5.3) 

Pn-lXn-l^qY Xn 

q&Q,Y&r 

Equation ()5.2|) can now be rewritten to 

-n/T/H I c . ( \\ P{IRun{pnX„)) 

P(V I Sat^.,iy)) = • S (5.4) 

where the meaning of 5 is given by equation 1)5. 3|) . So, 'PiVpx—'^n \ Satn-i{y)) is indeed 
independent of y, and hence equation l|5.4|) also defines the probability 'PiVpx Vn \ Satn~i)- 

□ 

Loosely speaking, the following lemma proves the memoryless property required to define 
a Markov chain: The probability of = Vn depends only on the value of , not on 

the values of F^^'^^ ' • • • ' ^pX • 

Lemma 5.11. The conditional probability of = Vn on the hypothesis = vi A 

■ • • A = Vn-i is equal to the probability of = Vn conditioned on = Vn-i, 
assuming that the probability of = wi A ■ ■ ■ A vj^"^ ""^^ = Vn^i is non-zero. 

Proof. The result follows immediately from Equation (|5.4() in the proof of Lemma fS.lOl The 
right side on the equation does not depend on the values of • • • , V^'^ . □ 
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Finally, as another consequence of Lemma f5.ini we obtain that the probability of = 
Vn does not depend on n: 

Lemma 5.12. The conditional probability of = {q'Y',a') on the hypothesis = 
{qY,a) is equal to the conditional probability of V^y — {q'^'^o,') '^'^ hypothesis V^y — 
{qY,ao), assuming that ViVpx = ilYja)) / 0. Moreover, the hypothesis that a run w 
satisfies V^y{w) = {qY,aQ) is the same as the hypothesis that w € IRun{qY). 

Proof. The first part follows immediately from the fact that n appears only as an index in 
Equation (|5.4j) . For the second, observe that, by definition, a run w starting at qY satisfies 

V^Y i'^)—'!^ if (1) is infinite and (2) its first minimum has head qY . But (1) and the 
fact that all configurations of an infinite run have length 1 or greater imply that the first 
configuration of the run is also its first minimum, and so, since w starts at qY , they imply 
(2). So a run w starting at qY satisfies V^y—I^ iff it is infinite, i.e., iff w € IRun{qY). □ 

Example 5.13. In order to give some intuition for these results, and in particular for the 
proof of Lemma l5.1fl( consider the special case in which the initial configuration is pX for 
some p G P, X G r, and the observing automaton A has one single state. In this case, the 

in) 

automaton always makes the same observation, and so we can write Vpx ~ 1^ instead of 

^pX ~ (q^^o,). We wish to obtain an expression for V{Vpx=QY). By the second part of 
Lemma 15.121 we have 

'P{V^j^=pX) = V{IRun{pX)) 

and therefore 

nySl=^y) = nviS=qY \ v^^j=px) • v{iRun{px)) 

Now we can apply equation 15.31 in the proof of Lemma I5.1fll and obtain 



V{V^'^2=qY) = V{IRun{qY)) ■ S 



and, by Equation 15.21 



p(l/^)=gy) = x-V{IRun{qY)) + 

pX^qY 

Y x-V{IRun{qY)) + 

pX^qYZ 

zer 

Y x-V{rZ,{QyiT*)U{qe])-V{IRun{qY)) (5.5) 

pX^rZY 

reQ,zer 

Let us interpret this equation. In order to reach the second minimum at qY there are 
only three possibilities for the first move. The first possibility is to move directly from pX 
to qY; in this case we must continue with any run that never terminates, since every infinite 
run of the form pX; qY; ■ ■ ■ necessarily has qY as second minimum. The probability of this 
case is captured by the first summand of Equation 15.51 The second possibility is to move 
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from pX to qY Z for some Z G F; in this case we must continue with an infinite run in 
which the stack content always has at least length 2, i.e., with a run of the form 

pX; qYZ; qiOiZ; . . . ; qiOiZ; . . . 

where all the a's are nonempty. This gives the second summand. Finally, the third possi- 
bility is to move from pX to rZY for some r € P, Z € T; we must then continue with a run 
that eventually "pops the Z" while entering state q, i.e., with a run of the form 

pX; rZY; riaiY; . . . ; rnttnY; qY; qif3i; . . . ; qi(3i; . . . 

where all the a's and /J's are nonempty. This gives the third summand. □ 

Lemma 15.111 and 15.121 allow us to define the finite Markov chain M/\. 

Definition 5.14. The finite-state Markov chain Ma has the following set of states 



One can readily check that Ma is indeed a Markov chain, i.e., for every state s of Ma 
we have that the sum of probabilities of all outgoing transitions of s is equal to one. Observe 
also that if both (qY, a) and {qY, a') are states of Ma, then they have the "same" outgoing 
arcs (i.e., {qY,a) ^ {rZ,a) iff {qY,a') {rZ,a), where x > 0). 

Example 5.15. We construct the Markov Chain Ma for the pBPA A of Figure H and 
the observing automaton A of Figure IH In fact, as we shall see, the states and transition 
probabilities of the chain depend on the value of the parameter x. 

Since the pBPA has one single control state, we omit it. The set of heads is then 
'H(A) = {Z,I,D} and the set of states of the observing automaton is A = {ao,ai}. In 
order to determine the states of the Markov chain we have to compute the pairs (Y, a) such 

that V{V^^^ = (y,a)) > 0. Recall the definition ofV{V^^^ = {Y,a)). This is the probability 
of, starting at the configuration Y, executing an infinite run such that (i) the head of the 
first minimum is Y, and (ii) the first observation of A is the state a. Since the initial 
configuration Y has the shortest possible length in an infinite run, (i) always holds. So 

V{\y^^ = {Y,a)) is the probability of executing an infinite run such that (ii) holds. Recall 
that the first observation of an observing automaton is the state it reaches after reading 
the sequence of heads between the first and the second minimum, excluding the first, but 
including the second. In the case of the automaton A of Figure |1J the first observation is 
oo if the sequence of heads does not contain the head Z, and oi otherwise. 

The values of ViV^ = {X, a)) for X £ {Z, I, D} and a G {cq, ai} are as follows: 





min{2x, 2 — 2x} if X = Z and a = ai 

max{0, {2x — l)/x} if X = I and a = 

max{0, (1 — 2x)/(l — x)} if X = D and a = 



otherwise 
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These values can be obtained using the definitions, but in this simple case we can also 
use more direct methods. Consider for instance ViV^^ = {Z,ai)). This is the probability 
of, starting at Z, executing an infinite run and visiting again a configuration with head Z 
before reaching the second minimum. Observe that all runs that start at Z are infinite, that 
the only configuration they visit with head Z \s Z itself, and that Z is always a minimum. So 
Viy^^ = {Z,ai)) is the probability of, starting at the configuration Z, eventually reaching 
Z again. This probability is equal to x ■ [/, e] + {1 — x) ■ where [I, e] and [D^e] are 

defined in Example 13.61 We get 

V{vP = {Z,ai)) = x-[I,e] + {l-x)-[D,e] 

= x ■ min{l, (1 — x)/x} + {1 — x) ■ min{l, x/(l — x)} 
= min{22;, 2 — 2x} 

Observe that the states of Ma depend on x. The states are _L, Z, I, D and 

{D, oo) ii X = 0, 

{Z,ai),{D,ao) if0<x<l/2, 
{Z,ai) ifx = l/2, 

{Z,ai),{I,ao) ifl/2<x<l, 
(/, ao) if x = 1. 

The Markov chain for the cases x = 1/2 and 1/2 < x < 1 are shown in Figure El 



2-2x 




Figure 5: The Markov chain Ma for x = 1/2 (left) and for 1/2 < x < 1 (right) 

Let us obtain the transition probability from (/, ao) to itself in the case 1/2 < x < 1. 
According to Definition 15. 14| the probability is equal to V{Vj'^^ = (/, oq) | Vj^^ = (/, oq)), 
i.e., to the probability of, assuming the first minimum has head /, reaching the second 
minimum at head I again, visiting no configuration with head Z in-between. Let us see 
that this probability is 1. If the first minimum is la for some a G {Z,I,D}*, then all 
subsequent configurations of the run are of the form (3a for a nonempty /3 (notice that we 
assume that the run is infinite, because finite runs have no minima). So /? must have head 
/ and so, in particular, the next minimum will also have head /. □ 
Not every run of A is "represented" in the Markov chain Ma . Consider for instance the 
case X = 1/2 and its corresponding chain Ma on the left of Figure |SJ Every configuration 
of the run Z; IZ; HZ; IIIZ; ... is a minimum, but its sequence of heads, i.e., ZI'^ , does not 
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correspond to any path of Ma- We show, however, that the "not represented" runs have 
probabihty 0. 

A trajectory in Ma is an infinite sequence (t(0)o"(1) • • • of states of Ma, where for 
every i G Nq, Prob{a{i) a{i + 1)) > 0. To every run w G Run{pX) of A we associate its 
footprint, denoted a^,, which is an infinite sequence of states of Ma defined as follows: 

• o-«)(0) = pX 

• if ID is finite, then for every i G N we have (Tu,(z) = _L; 

• if is infinite, then for every i G N we have (T^(i) = {piXi, Obsi{w)), where piXi is the 
head of mmi{w). 

We say that a given w G Run{pX) is good if a^j is a trajectory in Ma- Our next lemma 
reveals that almost all runs are good. 

Lemma 5.16. Let pX G 7i{A), and let Good be the subset of all good runs of Run(pX). 
Then V {Good) = 1. 

Proof. Let Bad = Run{pX) \ Good. Let Fail be the set of all finite sequences vq - ■ ■ f j+i of 
states of Ma such that i G Nq, vq = pX, vq - ■ ■ Vi is a trajectory in Ma, and Prob{vi — > 
= 0, where Prob is the probability assignment of Ma- Each y G Fail determines a 
set Bady = {w G Bad \ aw starts with y}. Obviously, Bad = ^y^paii^'^'^y prove that 
V{Bady) = for each y G Fail. Let y = vq - ■ ■ "yj+i. By applying definitions, we obtain 

V{Bady) = V{vi'J=v,A---AVi'^'^=v.,+^) 

Since 'P{Vpx=Vi A • • • A Vpx='t'i) / 0, the last fraction makes sense and it is equal to 

Prob{vi Vj+i) 

ViV^i=v.A---AV^'J=v,) 

which equals zero. □ 

5.3. V{Run{pX, Acc)) is effectively definable in (M, +,*,<). Recall that our aim is 
to show that V{Run{pX, Acc)) is effectively definable in (M, +, *, <). We will achieve this 
in Theorem 15.221 as an easy corollary of Lemma l5.2(JI This lemma states that V{pX,Acc) 
is the probability of, starting at pX, hitting so-called accepting bottom strongly connected 
component of Ma. As usual, a strongly connected component of Ma is a maximal set 
of mutually reachable states, and bottom strongly connected components are those from 
which no other strongly connected components can be reached. 

Definition 5.17. Let C be a bottom strongly connected component of Ma. We say that 
C is accepting if C / {_L} and the set {a £ A \ {qY,a) G C for some qY G 7i{A)} is an 
element of Acc (remember that Acc is the acceptance set introduced after Definition 15. 4|) . 
Otherwise, C is rejecting. 

We say that a given pair (qY, a), where qY G 'H{A) and a G ^, is recurrent, if it belongs 
to some bottom strongly connected component of Ma- 

We say that a run w G Run{pX) hits a pair {qY,a) G 7i{/S)^A if there is some i G N 
such that the head of minj(u;) is qY and Obsi{w) = a. The next lemma says that an infinite 
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run eventually hits a recurrent pair. In this lemma and the next we use the following well- 
known results for finite Markov chains (see e.g. |Fel66j ) : 

• A run visits some bottom strongly connected component of the chain with probability 1. 

• If a run visits some state of a bottom strongly connected component C, then it visits all 
states of C infinitely often with probability 1. 

Lemma 5.18. Let us assume that V{IRun{pX)) > 0. Then the conditional probability that 
w G Run{pX) hits a recurrent pair on the hypothesis that w is infinite is equal to one. 

Proof. Let Rec denote the event that a run of Run{pX) hits a recurrent pair. Due to 
Lemma l5.16( we have that 

ViRec I IRun{pX)) = V{Rec \ IRun{pX) n Good) (5.6) 

A run belongs to IRun{pX) n Good iff its footprint is a trajectory in Ma that does not 
hit the state _L. A run w G IRun{pX) D Good satisfies Rec iff its footprint hits (some) 
recurrent pair {qY, a) . It follows directly from the definition of Ma that the right-hand side 
of equation 1)5.6(1 is equal to the probability that a trajectory from pX in Ma hits a bottom 
strongly connected component on the hypothesis that the state _L is not visited. Since Ma 
is finite, this happens with probability one. □ 

So, an infinite run eventually hits a recurrent pair. Now we prove that if this pair 
belongs to an accepting/rejecting bottom strongly connected component of Ma, then the 
run will be accepting/rejecting with probability one. 

Lemma 5.19. The conditional probability that w G Run{pX) is accepting /rejecting on the 
hypothesis that the first recurrent pair hit by w belongs to an accepting /rejecting bottom 
strongly connected component of M/^ is equal to one. 

Proof. The argument is similar as in the proof of Lemma l5.18l Let C be a bottom strongly 
connected component of Ma- By ergodicity, the conditional probability that an infinite 
trajectory in Ma hits each state of C infinitely often on the hypothesis that the trajectory 
hits C is equal to one. □ 

A simple consequence of Lemma 15.191 is: 

Lemma 5.20. (cf. Proposition 4.1.5 of |CY95j ) Let pX G W(A). V{pX,Acc) is equal to 
the probability that a trajectory from pX in Ma hits an accepting bottom strongly connected 
component of Ma ■ 

Example 5.21. Consider the pBPA of Figure ^ and the observing automaton of Figure 0] 
V{Z, Acc) is the probability of, starting at Z, executing a run that visits configurations with 
head Z infinitely often. In the case x = 1/2, the bottom strongly connected components 
of Ma are {_L} and {(Z, ai)}, which are rejecting and accepting, respectively. Starting at 
the state Z of Ma, the probability of hitting {(Z, ai)} is 1, and so 'P{Z,Acc) = 1. In the 
case 1/2 < X < 1, the bottom strongly connected components of Ma are {_L} and {(/, ao)}, 
which are both rejecting, and so V{Z, Acc) = 0. 

Since the probability of hitting a given bottom strongly connected component of a given 
finite-state Markov chain is effectively definable in (M, +, *, <) by the results of Section EJ 
and the transition probabilities in Ma are well-definable too, we can conclude the following: 
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Theorem 5.22. V{Run{pX, Acc)) is effectively expressible in (M, +,*,<). In particular, 
for every rational constant y and every ~ G {<, <, >, >, =} there effectively exists a formula 
of (M, +, *, <) which holds iff V{Run{pX, Acc)) ~ y. 

5.4. Decidability of tj-regular properties. As a simple corollary of Theorem l5.221 we 

obtain the decidability of the qualitative/quantitative model-checking problem for pPDA 
and w-regular properties. Recall that a language of infinite words over a finite alphabet is 
uj-regular iff it can be accepted by a (deterministic) Muller automaton. 

Definition 5.23. A deterministic Muller automaton is a tuple B = {T,,B, g,bi,J^), where 
S is a finite alphabet, B is a finite set of states, g: B xT, ^ B is a (total) transition function 
(we write b b' instead of g{b,a) = b'), bj is the initial state, and C 2^ is a set of 
accepting sets. 

An infinite word w over the alphabet S is accepted by B if Inf{w) G J-, where Inf{w) 
is the set of all 6 G that appear infinitely often in the unique run of B over the word w. 

We consider specifications given by Muller automata having as their alphabet. It 

is well known that every LTL formula whose atomic propositions are interpreted over simple 
sets can be encoded into a deterministic Muller automaton having 'H(A) as alphabet. Our 
results can be extended to atomic propositions interpreted over arbitrary regular sets of 
configurations using the same technique as in 

Let us fix a deterministic Muller automaton B = (Ti.{A), B, g, bj,J^). An infinite run w 
of 7a is accepted by B if the associated sequence of heads of configurations in w is accepted 
by B. Let Run{pX, B) be the set of all w G Run{pX) that are accepted by B. We show 
that Run{pX, B) is effectively expressible in (M, +, *, <), and so we can decide if it is larger 
than, smaller than, or equal to some threshold p. 

Loosely speaking, we proceed as follows. We compute the synchronized product A' of 
A and B. Then, we define a A'-observing automaton A whose states are sets of states of 
B. The automaton observes heads of A', which are of the form {p, b)X, where pX is a 
head of A and 6 is a state of b. At the end of a "jump" , A returns the set of states of B 
that were visited during the jump. Hence, the observation Obs{w) of the automaton on a 
run w is a sequence B1B2 ■ ■ ■ of sets of states of B containing full information about which 
states were visited in which jump. Now it is just a matter of setting the acceptance set of 
A adequately: The acceptance sets of A are the sets {61, . . . , 6„} of states of A such that 
the union 61 U . . . U 6„ is an element of 

Theorem 5.24. V {Run{pX , B)) is effectively expressible in (M, +, *, <). In particular, for 
every rational constant y and every ~ G {<,<,>,>,=} there effectively exists a formula 
of (M, +,*,<) which holds iff V{Run{pX,B)) ~ y. (Hence, for each < A < 1 we can 
compute rationals , "P" such that < V{pX, Acc) < "P" and "P" — P^' < \.) 

Proof. Let A' = {QxB, F, 6', Prob') be the synchronized product of A and B, i.e., {p, b)X 
{t,b')a is a rule of A' iff pX ^ ta is a rule of A and g{b,pX) = b' . Consider the A'- 
observing automaton A = {A, ^,1, Acc) where ^ = 2^, oq = 0, C{M, {p, b)Y) = MU{b} for 
all M C i? and (p, b)Y G Tl{A'), and Acc is defined as follows: for every ai, . . . ,a„ G 2^, 
{ai, . . . , an} G Acc iff ai U . . . U a„ G .7-". 
It is easy to check that 

V{Run{pX, B)) = V{Run{{p, bi)X, Acc)) 
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Now it suffices to apply Theorem 15.221 □ 

6. Conclusions 

We have provided model checking algorithms for probabilistic pushdown automata 
against PCTL specifications, and against w-regular specifications represented by Muller 
automata. Contrary to the case of probabilistic finite automata, qualitative properties (i.e., 
whether a property holds with probability or 1), depend on the exact values of transition 
probabilities. 

There are many possibilities for future work. An obvious question is what is the com- 
plexity of the obtained algorithms. Of course, this depends on the complexity of the corre- 
sponding fragments of first order arithmetic of reals. It is known that the fragment obtained 
by fixing the alternation depth of quantifiers is decidable in exponential time [(xriSSj . and 
that the existential fragment (and hence also the universal fragment) is decidable even in 
polynomial space |('an88j . The formulas constructed in Section 01 have a fixed alternation 
depth, and so we can conclude that the qualitative/quantitative random walk problem is 
decidable in exponential time. Actually, we can do even better — if we are interested whether 
V{pX, Ci lA C2) < Q, we can simply ask if there is some solution of the corresponding system 
of quadratic equations (cf. Theorem 13. 5 j) such that the component of the solution which 
corresponds to V{pX,Cih( C2) is less than or equal to g. Obviously, the minimal solution 
(i.e., the probability of V{pX,CiU C2)) can only be smaller. Hence, the existential frag- 
ment is sufficient for deciding whether V{pX,CiU C2) < Q, and similarly we can use the 
universal fragment to decide whether V{pX,CiU C2) > Q- To sum up, the problem whether 
V{pX, Ci U C2) ~ Q, where ~ G {<, <, >, >, =}, is decidable in polynomial space. 

Recently, deeper results concerning the complexity of the reachability problem for pPDA 
and pBPA have been presented by Etessami and Yannakakis in |EY05j . In particular, they 
show that the qualitative reachability problem for pBPA processes (i.e., the question whether 
a given configuration is visited with probability 1) is decidable in polynomial time. It is also 
shown that the Square-Root-Sum problem (i.e., the question whether 'Y^^=i < c for a 
given tuple (ai, . . . , a„, c) of natural numbers) is polynomially reducible to the quantitative 
reachability problem for pBPA, and to the qualitative reachability problem for pPDA. The 
complexity of the Square- Root-Sum problem is a famous open problem in the area of 
exact numerical algorithms. It is known that the problem is solvable in polynomial space, 
but no lower bound (like NP or co-NP hardness) is known. This means that the PSPACE 
upper bound for the quantitative pBPA reachability and the qualitative pPDA reachability 
cannot be improved without achieving an improvement in the complexity of the Square- 
ROOT-SUM problem. 

Some of the problems which were left open in |EKM04] were solved later in |BKS05j . 
It was shown that the model-checking problems for PCTL and pPDA, and for PCTL* 
and pBPA, are undecidable (PCTL* is the probabilistic extension of CTL*). On the other 
hand, the decidability result about qualitative/quantitative model-checking pPDA against 
deterministic Biichi specifications was extended to Muller automata. In the qualitative case, 
the algorithm runs in time which is singly exponential in the size of a given pPDA and a 
given Muller automaton. In the quantitative case, the algorithm needs exponential space. 
Finally, it was shown that the model-checking problem for the qualitative fragment of the 
logic PECTL* and pPDA processes is also decidable. The complexity bounds are essentially 
the same as for Muller properties. 
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The complexity of model-checking w-regular properties (encoded by Biichi automata) 
for pPDA and pBPA processes was studied also in |EYj . The complexity bounds improve 
the ones given in |BKS05j . In particular, it is shown that the qualitative model-checking 
problem for pPDA and Biichi specifications is EXPTIME-complete. 

An interesting open problem is the decidability of the model-checking problem for 
PCTL and pBPA processes, i.e., whether there is an "exact" algorithm apart from the 
error-tolerant one given in Section 14.21 Another area of open problems is generated by 
considering model-checking problems for a more general class of pushdown automata whose 
underlying semantics is defined in terms of Markov decision processes (this model combines 
the paradigms of non-deterministic and probabilistic choice) . 
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